Conventionally, as illustrated in FIG. 15, with an image transmission system including a coding apparatus that displays video images acquired by a camera as real time moving images on a monitor, in order to reduce the amount of coding information, moving images are coded using a coding method in which I frames or B frames are inserted at regular intervals (see FIG. 16). In recent years, as illustrated in FIG. 17, by aiming at shortening the initial delay time (VBV Delay) for displaying video images on a monitor, a step-by-step refresh method has been used, in which an area (refresh area) subjected to an intra coding that processes a large amount of coding information is shifted for each frame (for example, see Japanese Laid-open Patent Publication No. 2005-124041).
Specifically, as illustrated in FIG. 18, with the coding method in which the I frames or the B frames are inserted at regular intervals, because the amount of coding information varies in each frame, a transmit/receive buffer having a large capacity needs to be designed for stable transmission and reception of a stream, thus needing a long initial delay time. In contrast, as illustrated in FIG. 19, with the coding method using the step-by-step refresh method, amount of coding information in each frame can be made uniform; therefore, a stable transmission and reception of a stream can be implemented without making the capacity of the transmit/receive buffer large. Accordingly, the amount of data stored in the transmit/receive buffer can be reduced, thus shortening the initial delay time until the commencement of decoding.
With the international telecommunication union telecommunication standardization sector (ITU-T) Rec.H.264/advanced video coding (AVC) (hereinafter, “H.264 coding method”), which is a typical coding technology in use today, when moving images are coded using the step-by-step refresh method, even though a deblocking filter process that removes distortion of the block boundary is performed, the distortion of the block boundary is not sufficiently removed.
Specifically, the deblocking filter process is used to smooth distortion of a block boundary by performing a predetermined calculation using pixel values of adjacent macroblocks. However, when the deblocking filter process is performed on a boundary between a refresh area in which a large amount of coding information is processed and a non-refresh area in which a small amount of coding information is processed, the distortion of the block boundary cannot be sufficiently removed because the pixel values of the macroblocks in the non-refresh area are used in the calculation for smoothing the macroblocks in the refresh area (hereinafter, in a calculation for smoothing macroblocks in a refresh area, using the pixel values of macroblocks in a non-refresh area is referred to as “leakage”).
In the example as illustrated in FIG. 20, the deblocking filter process is assumed to be performed on the block boundary (1) illustrated in FIG. 20, which is a boundary between a refresh area and a non-refresh area. The pixel values of the column (2) in FIG. 20 at the time the deblocking filter process is performed are illustrated in FIG. 21. FIG. 21 is a schematic diagram illustrating pixel values before the deblocking filter process and pixel values after the deblocking filter process is performed on the column (2) in FIG. 20. The pixel value “S0” after the deblocking filter process illustrated in FIG. 21 can be obtained using the equation represented by “S0=((s2×1)+(s1×2)+(s0×2)+(t0×2)+(t1×1)+4)/8”.
Specifically, the pixel value “S0” after the deblocking filter process is calculated using the pixel values “t0” and “t1” of the non-refresh area. When the pixel value “S0” of the refresh area is calculated using the pixel values “t0” and “t1” of the non-refresh area in which degradation of image is severe, distortion of the block boundary of the refresh area cannot be sufficiently removed.
When the distortion of the block boundary cannot be sufficiently removed using a deblocking filter, a motion compensation process is performed in a state in which distortion is present at the block boundary, which in turn causes image degradation to propagate to the next and subsequent frames.
To solve this problem, “leakage” can be prevented using a method described below. A first method is a method in which a deblocking filter processing function is not applied to all frames. With the H.264 coding method, it is possible to set the deblocking filter processing function by changing a parameter so that the function is not applied. Accordingly, “leakage” can be prevented by setting the deblocking filter processing function to not be applied only when a moving image is coded using the step-by-step refresh method.
A second method is a method in which the deblocking filter processing function is not applied to only the boundary between the refresh area and the non-refresh area. Specifically, a slice dividing function, which is used to divide a single frame into two or more frames, is applied to the boundary between the refresh area and the non-refresh area, and the deblocking filter processing function is set to be applied except a slice boundary. Accordingly, “leakage” can be prevented while the deblocking filter processing function is applied to areas other than the slice boundary.
FIG. 22 is a schematic diagram illustrating a method of preventing leakage using the slice dividing function. In the example illustrated in FIG. 22, a case is described in which a coding apparatus divides an input moving image signal into a plurality of frames 11 to 15 in time series and codes the frames this order: frame 11, frame 12, frame 13, . . . , frame 14, and frame 15. Another frame is present between the frame 13 and the frame 14; however, a description and a drawing thereof are omitted here.
As illustrated in FIG. 22, the coding apparatus divides each of frames 11 to 15 into macroblocks, then partitions the macroblocks into macroblock lines that are an assembly of macroblocks, and shifts down one macroblock line, which becomes a refresh area, for each successive frame. Specifically, a macroblock line 11a is determined to become the target of a refresh area in the frame 11, and a macroblock line 12a is determined to become the target of a refresh area in the frame 12, which is subsequently processed. In this way, in the example illustrated in FIG. 22, the coding apparatus codes the moving image using the step-by-step refresh method in which refresh areas are sequentially shifted down one macroblock line for each frame.
A macroblock line group 11b is partitioned into macroblock lines having the same height as that of the macroblock line 11a; however, an image of the partition is not illustrated in the figure and is referred to as a “macroblock line group” in this specification. This also applies to macroblock line groups 12c, 13b, 13c, 14b, and 15b in a similar manner.
In FIG. 22, the macroblock line indicated by “intra+inter” (for example, the macroblock line group 11b) indicates that it is coded, for each macroblock, using one of the intra coding and the inter coding, whichever processes a smaller amount of coding information. In general, the amount of coding information to be processed in the inter coding is smaller than that in the intra coding; however, with the H.264 coding method, there is a case in which the amount of coding information to be processed in the intra coding is smaller than that in the inter coding due to an intra prediction function being used (also called “intra-frame prediction”) in which compressibility is improved by using pixel correlation between blocks.
The intra prediction function has a plurality of prediction modes; the amount of coding information varies according to the prediction mode. When a refresh area is intra coded, the coding apparatus codes the refresh area in a prediction mode that exhibits the highest coding efficiency. When a non-refresh area is intra coded, the coding apparatus codes the non-refresh area in a prediction mode that processes a small amount of coding information.
When a moving image is coded using the step-by-step refresh method in this way, if a frame is subjected to the deblocking filter process, leakage from a non-refresh area, in which a small amount of coding information is processed, occurs into a refresh area, in which a large amount of coding information is processed. In the example illustrated in FIG. 22, in the frame 11, leakage from the macroblock line group 11b of the non-refresh area occurs into the macroblock line 11a of the refresh area.
To prevent such leakage, in the example illustrated in FIG. 22, the slice dividing function is applied to the boundary between the refresh area and the non-refresh area, and the deblocking filter process is performed on the frame 11 except the slice boundary. For example, in the frame 11, the boundary between the macroblock line 11a and the macroblock line group 11b is divided using the slice dividing function, and only a slice boundary 11s is not subjected to the deblocking filter process. Similarly, in the frame 12, only a slice boundary 12s is not subjected to the deblocking filter process.
However, with the conventional method in which the deblocking filter function is not applied to all the frames as described above, there is a problem in that distortion of the block boundary cannot be smoothed, even though smoothing is the purpose of the deblocking filter process. This leads to a problem of image degradation due to block distortion significantly appearing in the moving image to be decoded.
Furthermore, with the conventional method in which the slice dividing function is used and the deblocking filter processing function is applied to except the slice boundary, an image is degraded because overhead occurs due to the addition of a slice header. Specifically, a predetermined amount of information is needed to add the slice header, meaning the amount of information allocated to the coding needs to be reduced. Accordingly, the moving image to be decoded is degraded.
Furthermore, when the slice dividing function is used, compression efficiency of the coding is reduced because the intra prediction function cannot be applied to the frames that are divided using the slice dividing function. This means that efficient coding cannot be performed, thus causing a problem of degradation of the moving image to be decoded.
As described above, using the step-by-step refresh method, when the moving image is coded using the H.264 coding method or the like that has a deblocking filter processing function, an important issue is how to minimize the degree of degradation of the moving image to be decoded during the coding.